More References

References are listed on other pages, including here, here, and here.

The following references haven't been incorporated into any pages yet. These books are difficult to classify, as most fall on the interesting boundaries between Evolution, Design, Computation, and Intelligence.

Emergent Collective Intelligence

Dorigo, Marco, 2001. Ant Algorithms and Swarm Intelligence. Tutorial MP-1 at The Seventeenth International Joint Conference on Artificial Intelligence.

I attended this tutorial at IJCAI-2001; Dorigo and others have published numerous books and articles.

Dyson, George B., 1997. Darwin among the Machines: The Evolution of Global Intelligence.

Dyson focuses on ideas from Von Neumann, etc. from the early years of computing and their relevance for today. He gave a great presentation at Google and did a very interesting two-part write-up on afterwards: part I, part II.

Szuba, Tadeusz M., 2001. Computational Collective Intelligence.

The most mind-bending book on collective intelligence I have found. Wikipedia attempts to summarize it here. Szuba describes a Random Prolog Processor and asserts his parallel (social) probabilistic algorithms can be polynomial for NP problems (p. 56).

Computer Algorithms

With the huge rise in grid computing power and availability of large datasets the emphasis in Artificial Intelligence has drastically shifted to Statistical Machine Learning.

Battelle, John, 2005. The Search: How Google and its Rivals Rewrote the Rules of Business and Transformed our Culture.

Manning, Christopher D. and Hinrich Schutze, 1999. Foundations of Statistical Natural Language Processing.

Mitchell, Tom M., 1997. Machine Learning.

This book is good but dated; a new edition would probably focus more strongly on new statistical techniques.

How the Mind Works

Calvin, William H., 1996. The Cerebral Code: Thinking a Thought in the Mosaics of the Mind.

Language took over areas evolved to control timing in throwing. As a board gamer I love the centrality of the hexagon in his theory.

Hawkins, Jeff, with Sandra Blakeslee, 2004. On Intelligence.

Hawkins focuses on the use of feedback in pattern recognition and memory. He has founded a new startup Numenta to build hardware based on these ideas. The Redwood Neuroscience Institute researchers also developed these ideas; a video is available here.

Lakoff, George and Mark Johnson, 1999. Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought.

Lakoff and Johnson present three basic findings from cognitive science research (p. 3):

and examine their impact on Western Philosophy. Artificial Intelligence researchers such as Rodney Brooks also recognize their huge impact on A.I. They realize that we utilize our embodied experience of a three-dimensional world to reason about abstract concepts. We use metaphor to bridge (a metaphor :-) between the physical and abstract worlds. A "brain in a vat" A.I. has no experience of real-world relationships and cannot understand and utilize metaphors correctly. This realization helped spur new interest in humanoid robots such as Cog.

Minsky, Marvin, 1985. The Society of Mind.

Minsky proposes the mind operates as a huge collection (society) of simple agents each specializing on different parts of the problems of intelligence. The book itself is a collection of 270 interlinked one-page essays about these ideas. For years whenever someone posed a question in the newsgroup Minsky would answer with a reference to the appropriate numbered essay.

Minsky, Marvin, 2006. The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind.

Minsky expands on the role of emotions in his Society of Mind theory. Draft currently available on his home page

Pinker, Steven, 1997. How the Mind Works.

Evolutionary Theory

Kauffman, Stuart, 2000. Investigations.

Emergence and exploration of the adjacent possible: "In short, the known universe has not had time since the big bang to create all possible proteins of length 200 once." (p. 144)

Margulis, Lynn and Dorion Sagan, 2002. Acquiring Genomes: A Theory of the Origins of Species.

Margulis and Sagan radically argue that new species often arise not through mutation but instead through symbiotic merger of genomes: "the branches of animal evolutionary trees do not just branch but fuse." (p. 172) Fascinating examples include insects and sea slugs.

Dennett, Daniel C., 1995. Darwin's Dangerous Idea: Evolution and the Meanings of Life.

Origins of Consciousness, Language, Culture, and Technology

Burke, James and Robert Ornstein, 1997. The Axemaker's Gift: Technology's Capture and Control of our Minds and Culture.

Language is a technology used to describe how to make other technologies, like axes.

Calvin, William H., 2000. Lingua ex Machina: Reconciling Darwin and Chomsky with the Human Brain.

Language used for reciprocal altruism.

Deacon, Terrence W., 1997. The Symbolic Species: The co-evolution of language and the brain.

Proposes language was invented to make marriage work.

Dennett, Daniel C., 1986. "Julian Jayne's Software Archeology", in Brainchildren: Essays on Designing Minds, 1998.

Greenspan, Stanley I., and Stuart G. Shanker, 2004. The First Idea: How Symbols, Language and Intelligence Evolved from Our Primate Ancestors to Modern Humans.

Importance of early nurturing in passing on cultural knowledge.

Jaynes, Julian, 1976. The Origin of Consciousness in the Breakdown of the Bicameral Mind.

Alan Kay in Wired: "Did you ever read that book called The Origin of Consciousness in the Breakdown of the Bicameral Mind by Julian James? He claims that we didn't even get aware of consciousness until recently. It's the best book I've ever read that couldn't possibly be true."

Dennett, above, was also fascinated: "How could one take such a book seriously? Because it asked some very good questions that had never been properly asked before and boldly proposed answers to them." (p. 121)

Jaynes proposed that in early civilizations humans were not fully conscious and were instead ruled by inner voices. The large differences between the Iliad and Odyssey or the Old and New Testaments were not just stylistic but actually reflect changes in how humans experienced the world. Laments about the gods leaving were triggered by the withdrawal of the bicameral voices.

Pinker, Steven, 1994. The Language Instinct: How the Mind Creates Language.

Stewart, Ian and Jack Cohen, 1997. Figments of Reality: The Evolution of the Curious Mind.

Algorithmic Forms in Nature

Ball, Philip, 1999. The Self-Made Tapestry: Pattern Formation in Nature.

Flake, Gary William, 1998. The Computational Beauty of Nature: Computational Explorations of Fractals, Chaos, Complex Systems, and Adaptation.

Meinhardt, Hans, 2003. The Algorithmic Beauty of Seashells.

Wade, David, 2003. Li: Dynamic Form in Nature.

A small, short book of black and white pictures of fascinating shapes in nature.

Wolfram, Stephen, 2002. A New Kind of Science.

For a more radical view of the universe-as-cellular-automata-computation see Edward Fredkin's Digital Philosophy.


Alexander, Christopher, Sara Ishikawa and Murray Silverstein, 1977. A Pattern Language.

Alexander is an architect, and he specified a language of patterns usable by anyone to design buildings and spaces people would be comfortable living in. In Gamma et. al. software engineers appropriated the pattern concept for computer programming.

Alexander, Christopher, 1979. The Timeless Way of Building.

Alexander, Christopher, 2002. The Nature of Order: An Essay on the Art of Building and the Nature of the Universe. Volume 1: The Phenomenon of Life. Volume 2: The Process of Creating Life.

In the Nature of Order Alexander describes the principles underlying his patterns. He believes some design forms measurably contain more life. "I managed to identify fifteen structural features which appear again and again in things which do have life. These are (p. 144):

For Alexander an artificial structure contains life when it is created using the same types of recursively composed, algorithmic processes used by nature. Thus his books are closely tied to the others about algorithmic forms in nature.

Gamma, Erich, Richard Helm, Ralph Johnson, and John Vlissides, 1995. Design Patterns: Elements of Reusable Object-Oriented Software.

Code Generation

Code generation is a programming technique whereby a high-level declarative specification (genotype) of a system is turned into working code (phenotype). The high-level specification can be done in a domain specific mini-language optimized for the application. The high-level specification, which forms the "DNA" of the application, can also be manipulated using evolutionary algorithms. Note the repetition of the language and pattern concepts from Alexander.

Czarnecki, Krzysztof, and Ulrich W. Eisenecker, 2000. Generative Programming: Methods, Tools, and Applications.

Greenfield, Jack and Keith Short, 2004. Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools.

Lipson, Hod, Erik K. Antonsson, and John R. Koza, cochairs, 2003. Computational Synthesis: From Basic Building Blocks to High Level Functionality. Papers from the 2003 AAAI Spring Symposium.

I attended this symposium. The presented papers formed an interesting mixture of domain-specific languages, code generation, and evolutionary techniques.


Mackay, Charles, 1841. Extraordinary Popular Delusions and the Madness of Crowds.

We haven't gotten any smarter.

Mandelbrot, Benoit and Richard L. Hudson, 2004. The (Mis)behavior of Markets: A Fractal View of Risk, Ruin, and Reward.

The inventor of fractals criticizes the use of normal (Gaussian) probability distributions to model price changes in financial markets.

Taleb, Nassim Nicholas, 2001. Fooled by Randomness: The Hidden Role of Chance in the Markets and Life.

Demonstrates the power of Monte Carlo (random number computer simulation) techniques in financial modeling.

The Singularity

Vinge introduced the singularity concept in his 1993 essay. He asserted that within 30 years we would create superhuman intelligence. As both a scientist and science fiction writer he found it quite difficult to project what would happen after this event.

Kurzweil, Ray, 2005. The Singularity is Near: When Humans Transcend Biology.

Presents key technologies leading up to the singularity, and why we shouldn't be worried. His key technologies are (from Table of Contents):

He maintains a detailed web site

Stross, Charles, 2005. Accelerando.

A great science fiction presentation of the events in the lives of the last generation of humans and the first generations of post-humans.

Vinge, Vernor, 1993. The Singularity.

Reconciliation between Science and the Humanities

Wilson, Edward O., 1998. Conscilience: The Unity of Knowledge.